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by Postdetector Nonlinearities 
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This article gives the performance degradation found by using a computer 
program to calculate the effects of various processing techniques applied 
after envelope detection of narrowband Gaussian noise plus a CW signal. 
Nonlinear processes that we studied are: suppression of low output levels, 
hard limiting of high output levels, restricted dynamic range of output 
level, and quantizing the detector output. 

By modifying the program, we produced performance curves for systems 
in which a steady CW component ivas present in addition to possible 
signals. We also adapted the program to generate performance curves for 
systems using a square-law detector so that a comparison could be made 
with the results obtained for an envelope detector. We found that the presence 
of a CW component, and all the nonlinear processes applied after envelope 
detection, caused a loss of sensitivity in detecting small CW signals. Quan- 
tizing even in as few as 8 levels, however, caused very little additional 
degradation. 

I. INTRODUCTION 

A computer program was developed that would draw receiver op- 
erating characteristic curves for a system using an envelope detector 
to search for CW signals in narrowband Gaussian noise. 1 The pro- 
gram computed the probability that the detector output would exceed 
a chosen threshold under two circumstances : 

(i) No signals were present; this result gives the probability of 
false alarm, P PA . 

(u) Signals at various S/N were present; this result gives the prob- 
ability of detection, P D . 

S/N is the ratio of the CW signal power to the noise power ac- 
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companying it in the specified narrow band. These probabilities were 
calculated using formulas which described the appropriate distribu- 
tions of the detector output. 

The program also averaged independent samples of the detector 
output, and produced appropriate receiver operating characteristic 
curves by deriving a Gram-Charlier A series for the distribution of 
the sample average. 

When the program was adapted to take into account the modifica- 
tions of the detector output caused by various subsequent nonlinear 
processes, receiver operating characteristic curves were produced for 
systems in which such processes occurred. The performance degrada- 
tion was determined as the change in S/N that would be required to 
make the detectability of a signal, for given probability of false alarm, 
the same as without the nonlinear process. 

II. DISCUSSION 

Since some types of nonlinear processing greatly reduce the labor 
of handling large amounts of data, and others are unavoidable, or 
their effects are costly to minimize, it is useful to know the penalties 
on performance that are incurred by their presence. 

The types of nonlinear process studied are: suppressing low output 
levels, hard limiting of high output levels, restricting dynamic range 
of the output level, and quantizing the output of the detector. 

Suppressing low output levels occurs in some kinds of recording 
equipment where the output must reach a minimum level before a 
record can be made. High output limiting always occurs because the 
power handling capacity of physical equipment is limited. Dynamic 
range is restricted when the first two processes occur together; this 
also represents a commonly used technique in which a fluctuating out- 
put is converted to a binary waveform with respect to a chosen 
threshold. Quantizing is particularly useful when a digital computer 
is used to implement the statistical analysis. With this application in 
mind quantizing was evaluated in combination with restricted dynamic 
range. 

For simplicity in programming, we assumed that the unit used to 
process the output of the envelope detector had an appropriate trans- 
fer characteristic from the set shown in Fig. 1. It can be seen that over 
part of its operating range this unit has a linear characteristic. Quan- 
tizing would cause the slope of the linear region in Fig. 1(c) to be 
replaced by a staircase. 

In addition to the changes needed to allow investigation of the 
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Fig. 1 — Transfer characteristics of post detector processor, (a) Suppression of 
low levels, (b) hard limiting of high levels, (c) restricted dynamic range. 



above processing techniques, we made alterations in the program so 
that it would produce receiver operating characteristic curves for sys- 
tems using a square-law detector, and for systems in which an un- 
wanted CW component occurred in the specified narrow band in ad- 
dition to the one representing a signal. 

The latter case is typical of situations where a large, relatively 
stable, CW component tends to mask the presence of a smaller stable 
signal in the resolved band of a spectrum analyser. The results show 
how effective in such cases is a decision process similar to that de- 
scribed in Ref. 1, for detecting a small signal. 

The distortions produced by some of these processes on the detector 
output distributions are so radical that many samples must be aver- 
aged to get an adequate Gram-Charlier representation even using up 
to 31 moments, the limit available in the program. Consequently a 
large number (8192) of samples were averaged before deriving the 
Gram-Charlier series from which the receiver operating characteristic 
curves could be produced. We made some checks where it was pos- 
sible to produce curves for fewer samples averaged. We marked these 
on the figures; they show only slight deviations from the results ob- 
tained by averaging 8192 samples. 

Since the normal distribution becomes a satisfactory approxima- 
tion as the number of samples averaged grows large it may be in- 
ferred that the degradation caused by nonlinear processes could be 
estimated by averaging enough samples and using the normal ap- 
proximation. For some of the processes described here the receiver 
operating characteristic curves for averaging 8192 samples showed 
perceptible variation from the results obtained by a normal approxi- 
mation at Pfa values less than 10~ 4 . Thus, to ensure the validity of 
a normal approximation it might be necessary to average a very large 
number of samples indeed. 

When the results for one of the simulated nonlinear processes were 
plotted on the appropriate chart of Ref. 1, the variation of the degra- 
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dation from the average was never found greater than 10 per cent and 
usually within a few per cent. Consequently the average degradation 
shown in the accompanying figures represents a good approximation 
within the range covered by the appropriate chart of Ref. 1. 

Fig. 2 shows the degradation in performance when low output levels 
are suppressed, and when high levels are hard-limited. The position 
at which the levels are truncated is given along the abscissa as a 
multiple of the mean value of the ideal detector output when only 
noise is present (Rayleigh mean). The ordinate gives the change in 
S/N required with the distorting process to give the same performance 
as can be achieved without it. 

Fig. 3 shows the degradation sustained when the dynamic range 
factor of the detector output is 10 (solid line) and 2 (broken line). 
The dynamic range factor is the ratio of the high end of the linear 
range to the low end for a transfer characteristic like that in Fig. 1(c). 
Fig. 3 also gives the degradation sustained when the output is vir- 
tually converted to a binary signal by limiting the dynamic range 
factor to 1.001 (dotted line). The lower truncation point is shown 
on the abscissa and the ordinate gives the degradation in dB. 

III. QUANTIZING 

When a digital computer is used to implement the statistical anal- 
ysis of the detector output, it is important to know the maximum 
number of bits that need to be used to encode the output level. Three 
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Fig. 2 — Degradation caused by (a) suppressing low output levels and (b) hard 
limiting high output levels. 
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Fig. 3 — Degradation resulting from limited dynamic range. 

bits allows coding into 8 levels, and four bits allows coding into 16 
levels, and so on. If three bits are used, for example, the range of 
possible detector outputs must be divided into 8 regions. The highest 
region will include all levels that would equal or exceed the high end 
of a characteristic such as that in Fig. lc, and the lowest region will 
include all levels below the low end. The remaining six regions will 
be between the two truncation points. The computer simulation was 
carried out by dividing the detector output in this way and assigning 
the value of the lower limit of any region to all the values in it less 
than the upper limit. 

From Fig. 3 it can be seen that if a dynamic range factor of 10 is 
used and the low end is placed at three tenths of the Rayleigh mean, 
very little degradation in performance is suffered. When this range 
was quantized into 6 equal steps it was found that the total degrada- 
tion was only about 0.01 dB more than the value shown in Fig. 3. 

Two other ways of dividing the range into 6 unequal steps were 
tried, but the results were poorer than that for equal steps. In the first 
of these the range between the low end and high end was divided into 
6 intervals in such a way that the change in probability density was 
the same between each. The degradation in performance was about 
0.1 dB greater than that using 6 equal steps. The other way of quan- 
tizing the range divided it into 6 intervals of constant change in 
cumulative probability. The degradation in this case was 0.39 dB 
greater than that using 6 equal steps. 



IV. CW INTRUDER 

Fig. 4 shows that the system performance falls off rapidly owing 
to the presence of a CW component as the level of such a component 
increases from —10 dB with respect to the noise in the narrow band. 
The distribution curves corresponding to the no-signal hypothesis in 
this case were those for Gaussian noise plus a CW component at the 
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Fig. 4 — Degradation resulting from added CW signal. 

S/N specified on the abscissa. The curves corresponding to the hypoth- 
esis that a signal was present were those for Gaussian noise plus 
a resultant sine wave component. The power of the resultant equalled 
the power of the small signal plus the power of a large CW compo- 
nent at the level specified on the abscissa. The results thus represent 
the average for all possible phase relationships between the CW 
component and small signal assuming that all phase values are equally 
likely, and many samples of the detector output are averaged. 



V. SQUARE-LAW DETECTOR 

It was found that satisfactory Gram-Charlier series approxima- 
tions to the output distribution of a square-law detector could be 
produced only when a few hundred or more samples were averaged. 
Consequently subprograms were written to compute values for Chi- 
square and noncentral Chi-square distributions over the desired ranges 
of S/N and samples averaged. Using these and the results reported in 
Ref. 1 it was possible to produce the curves shown in Figs. 5, 6, and 7. 
which enable the receiver operating characteristic curves of Ref. 1 
to be used to estimate the performance for a square-law detector 
with quite good accuracy. Fig. 5 compares the performance of linear 
and square-law detectors at a false-alarm probability of 10 -8 and 
three different detection probabilities when the number of samples 
averaged ranges from 1 to 8192. The approximate S/N corresponding 
to a P D , Pfa pair is also given over the range of samples averaged. 
Curve 1 in Figs. 6 and 7 compares the detector performance for the 
same S/N as curve 1 in Fig. 5, but when the P FA is 5 X 10 -4 , and 0.09, 
respectively, and curve 2 in Figs. 6 and 7 do the same for the S/N 
that applies to curve 2 in Fig. 5, 
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Fig. 5 — Comparison of linear and square-law detectors. S/N (dB) in filter 
bandwidth, for P D = (*), P M = 10"°. 

VI. CONCLUSION 

We have given curves which show that the sensitivity for detecting 
small signals using an envelope detector is degraded when any of 
several common kinds of nonlinear processes occur between the detec- 
tor output and the averager used prior to the decision threshold. 
When only one sample of the detector output is used to form a deci- 
sion, the system will be oblivious to the presence of the nonlinear 
process as long as the decision threshold lies within the linear range. 
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Fig. 7 — Comparison of linear and square-law detectors, Pfa = 0.09. 

The system performance in such a case is thus restricted to threshold 
variations within this range. 

It is interesting that J. V. Harrington has analyzed the detection 
of repeated signals in noise using binary integration. 2 The system he 
assumed corresponds closely to the one discussed here for which Fig. 
3 shows the results. Harrington deduced that the optimum position 
for the quantizing threshold was about 1.44 times the Rayleigh mean, 
in which case binary integration would give results 0.77 dB poorer 
than if ideal processing with a linear detector had been used. These 
numbers are in excellent agreement with the minimum degradation 
condition shown in Fig. 3. 

We have shown that the presence of an unwanted CW signal, even 
— 10 dB with respect to the narrow-band noise, causes some degrada- 
tion of the sensitivity for detecting small CW signals, and the degrada- 
tion increases rapidly as the level of the unwanted component rises 
from there. 

Computation shows that there is little performance difference to 
be expected from the use of envelope or square- law detectors. Envelope 
detectors would thus seem preferable in view of their greater simpli- 
city, especially when a considerable dynamic range is to be covered. 
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